127 research outputs found
IIFL: Implicit Interactive Fleet Learning from Heterogeneous Human Supervisors
Imitation learning has been applied to a range of robotic tasks, but can
struggle when (1) robots encounter edge cases that are not represented in the
training data (distribution shift) or (2) the human demonstrations are
heterogeneous: taking different paths around an obstacle, for instance
(multimodality). Interactive fleet learning (IFL) mitigates distribution shift
by allowing robots to access remote human teleoperators during task execution
and learn from them over time, but is not equipped to handle multimodality.
Recent work proposes Implicit Behavior Cloning (IBC), which is able to
represent multimodal demonstrations using energy-based models (EBMs). In this
work, we propose addressing both multimodality and distribution shift with
Implicit Interactive Fleet Learning (IIFL), the first extension of implicit
policies to interactive imitation learning (including the single-robot,
single-human setting). IIFL quantifies uncertainty using a novel application of
Jeffreys divergence to EBMs. While IIFL is more computationally expensive than
explicit methods, results suggest that IIFL achieves 4.5x higher return on
human effort in simulation experiments and an 80% higher success rate in a
physical block pushing task over (Explicit) IFL, IBC, and other baselines when
human supervision is heterogeneous
Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Features
Humans make extensive use of vision and touch as complementary senses, with
vision providing global information about the scene and touch measuring local
information during manipulation without suffering from occlusions. While prior
work demonstrates the efficacy of tactile sensing for precise manipulation of
deformables, they typically rely on supervised, human-labeled datasets. We
propose Self-Supervised Visuo-Tactile Pretraining (SSVTP), a framework for
learning multi-task visuo-tactile representations in a self-supervised manner
through cross-modal supervision. We design a mechanism that enables a robot to
autonomously collect precisely spatially-aligned visual and tactile image
pairs, then train visual and tactile encoders to embed these pairs into a
shared latent space using cross-modal contrastive loss. We apply this latent
space to downstream perception and control of deformable garments on flat
surfaces, and evaluate the flexibility of the learned representations without
fine-tuning on 5 tasks: feature classification, contact localization, anomaly
detection, feature search from a visual query (e.g., garment feature
localization under occlusion), and edge following along cloth edges. The
pretrained representations achieve a 73-100% success rate on these 5 tasks.Comment: RSS 2023, site: https://sites.google.com/berkeley.edu/ssvt
Semantic Mechanical Search with Large Vision and Language Models
Moving objects to find a fully-occluded target object, known as mechanical
search, is a challenging problem in robotics. As objects are often organized
semantically, we conjecture that semantic information about object
relationships can facilitate mechanical search and reduce search time. Large
pretrained vision and language models (VLMs and LLMs) have shown promise in
generalizing to uncommon objects and previously unseen real-world environments.
In this work, we propose a novel framework called Semantic Mechanical Search
(SMS). SMS conducts scene understanding and generates a semantic occupancy
distribution explicitly using LLMs. Compared to methods that rely on visual
similarities offered by CLIP embeddings, SMS leverages the deep reasoning
capabilities of LLMs. Unlike prior work that uses VLMs and LLMs as end-to-end
planners, which may not integrate well with specialized geometric planners, SMS
can serve as a plug-in semantic module for downstream manipulation or
navigation policies. For mechanical search in closed-world settings such as
shelves, we compare with a geometric-based planner and show that SMS improves
mechanical search performance by 24% across the pharmacy, kitchen, and office
domains in simulation and 47.1% in physical experiments. For open-world real
environments, SMS can produce better semantic distributions compared to
CLIP-based methods, with the potential to be integrated with downstream
navigation policies to improve object navigation tasks. Code, data, videos, and
the appendix are available:
https://sites.google.com/view/semantic-mechanical-searc
FogROS2-SGC: A ROS2 Cloud Robotics Platform for Secure Global Connectivity
The Robot Operating System (ROS2) is the most widely used software platform
for building robotics applications. FogROS2 extends ROS2 to allow robots to
access cloud computing on demand. However, ROS2 and FogROS2 assume that all
robots are locally connected and that each robot has full access and control of
the other robots. With applications like distributed multi-robot systems,
remote robot control, and mobile robots, robotics increasingly involves the
global Internet and complex trust management. Existing approaches for
connecting disjoint ROS2 networks lack key features such as security,
compatibility, efficiency, and ease of use. We introduce FogROS2-SGC, an
extension of FogROS2 that can effectively connect robot systems across
different physical locations, networks, and Data Distribution Services (DDS).
With globally unique and location-independent identifiers, FogROS2-SGC securely
and efficiently routes data between robotics components around the globe.
FogROS2-SGC is agnostic to the ROS2 distribution and configuration, is
compatible with non-ROS2 software, and seamlessly extends existing ROS2
applications without any code modification. Experiments suggest FogROS2-SGC is
19x faster than rosbridge (a ROS2 package with comparable features, but lacking
security). We also apply FogROS2-SGC to 4 robots and compute nodes that are
3600km apart. Videos and code are available on the project website
https://sites.google.com/view/fogros2-sgc.Comment: 9 pages, 8 figure
Improving stamina and mobility with preop walking in surgical patients with frailty traits -OASIS IV: randomized clinical trial study protocol
BACKGROUND: Frail older surgical patients face more than a two-fold increase in postoperative complications, including myocardial infarction, deep vein thrombosis, pulmonary embolism, pneumonia, ileus, and others. Many of these complications occur because of postoperative loss of stamina and poor mobility. Preoperative exercise may better prepare these vulnerable patients for surgery. We present the protocol for our ongoing randomized trial to assess the impact of a preoperative walking intervention with remote coaching and pedometer on outcomes of stamina (six-minute walk distance- 6MWD) and mobility (postoperative steps) in older adults with frailty traits.
METHODS: We will be conducting a randomized clinical trial with a total of 120 patients permitting up to a 33% rate of attrition, to reach a final sample size of 80 (with 40 patients for each study arm). We will include patients who are age 60 or higher, score 4 or greater on the Edmonton Frailty Scale assessment, and will be undergoing a surgical operation that requires a 2 or more night hospital stay to be eligible for our trial. Using block randomization stratified on baseline 6MWD, we will assign patients to wear a pedometer. At the end of three baseline days, an athletic trainer (AT) will provide a daily step count goal reflecting a 10-20% increase from baseline. Subsequently, the AT will call weekly to further titrate the goal or calls more frequently if the patient is not meeting the prescribed goal. Controls will receive general walking advice. Our main outcome is change in 6MWD on postoperative day (POD) 2/3 vs. baseline. We will also collect 6MWD approximately 4 weeks after surgery and daily in-hospital steps.
CONCLUSION: If changes in a 6MWD and step counts are significantly higher for the intervention group, we believe this will confirm our hypothesis that the intervention leads to decreased loss of stamina and mobility. Once confirmed, we anticipate expanding to multiple centers to assess the interventional impact on clinical endpoints.
TRIAL REGISTRATION: The randomized clinical trial was registered on clinicaltrials.gov under the identifier NCT03892187 on March 27, 2019
- …